Goto

Collaborating Authors

 Northwestern Federal District


Extraction of informative statistical features in the problem of forecasting time series generated by It{ô}-type processes

Korolev, Victor, Ivanov, Mikhail, Kukanova, Tatiana, Rukavitsa, Artyom, Vakshin, Alexander, Solomonov, Peter, Zeifman, Alexander

arXiv.org Machine Learning

In this paper, we consider the problem of extraction of most informative features from time series that are regarded as observed values of stochastic processes satisfying the It{ô} stochastic differential equations with unknown random drift and diffusion coefficients. We do not attract any additional information and use only the information contained in the time series as it is. Therefore, as additional features, we use the parameters of statistically adjusted mixture-type models of the observed regularities of the behavior of the time series. Several algorithms of construction of these parameters are discussed. These algorithms are based on statistical reconstruction of the coefficients which, in turn, is based on statistical separation of normal mixtures. We obtain two types of parameters by the techniques of the uniform and non-uniform statistical reconstruction of the coefficients of the underlying It{ô} process. The reconstructed coefficients obtained by uniform techniques do not depend on the current value of the process, while the non-uniform techniques reconstruct the coefficients with the account of their dependence on the value of the process. Actually, the non-uniform techniques used in this paper represent a stochastic analog of the Taylor expansion for the time series. The efficiency of the obtained additional features is compared by using them in the autoregressive algorithms of prediction of time series. In order to obtain pure conclusion that is not affected by unwanted factors, say, related to a special choice of the architecture of the neural network prediction methods, we used only simple autoregressive algorithms. We show that the use of additional statistical features improves the prediction.


The man who ruined mathematics

New Scientist

Gödel's seminal work directly contradicted one of the great minds of mathematics and limited the field forever Kurt Gödel, the man who ruined mathematics, was one of the most important thinkers of the 20th century. He was born in 1906, smack-bang in the middle of the greatest crisis that maths has ever known. Just a few decades later, he would help resolve this turmoil, but in doing so doom mathematicians to a smaller world than the one that came before. Mathematics, as an intellectual framework, is incredibly powerful. The entire point is taking one set of logical ideas and using them to build another, making maths the closest thing we have to a cognitive perpetual-motion machine - there is always a new mathematical idea lurking across the horizon, and we just need to assemble the steps to get there.


Time-Warping Recurrent Neural Networks for Transfer Learning

Hirschi, Jonathon

arXiv.org Machine Learning

Dynamical systems describe how a physical system evolves over time. Physical processes can evolve faster or slower in different environmental conditions. We use time-warping as rescaling the time in a model of a physical system. This thesis proposes a new method of transfer learning for Recurrent Neural Networks (RNNs) based on time-warping. We prove that for a class of linear, first-order differential equations known as time lag models, an LSTM can approximate these systems with any desired accuracy, and the model can be time-warped while maintaining the approximation accuracy. The Time-Warping method of transfer learning is then evaluated in an applied problem on predicting fuel moisture content (FMC), an important concept in wildfire modeling. An RNN with LSTM recurrent layers is pretrained on fuels with a characteristic time scale of 10 hours, where there are large quantities of data available for training. The RNN is then modified with transfer learning to generate predictions for fuels with characteristic time scales of 1 hour, 100 hours, and 1000 hours. The Time-Warping method is evaluated against several known methods of transfer learning. The Time-Warping method produces predictions with an accuracy level comparable to the established methods, despite modifying only a small fraction of the parameters that the other methods modify.


Characterization of Gaussian Universality Breakdown in High-Dimensional Empirical Risk Minimization

Yaakoubi, Chiheb, Louart, Cosme, Tiomoko, Malik, Liao, Zhenyu

arXiv.org Machine Learning

We study high-dimensional convex empirical risk minimization (ERM) under general non-Gaussian data designs. By heuristically extending the Convex Gaussian Min-Max Theorem (CGMT) to non-Gaussian settings, we derive an asymptotic min-max characterization of key statistics, enabling approximation of the mean $μ_{\hatθ}$ and covariance $C_{\hatθ}$ of the ERM estimator $\hatθ$. Specifically, under a concentration assumption on the data matrix and standard regularity conditions on the loss and regularizer, we show that for a test covariate $x$ independent of the training data, the projection $\hatθ^\top x$ approximately follows the convolution of the (generally non-Gaussian) distribution of $μ_{\hatθ}^\top x$ with an independent centered Gaussian variable of variance $\text{Tr}(C_{\hatθ}\mathbb{E}[xx^\top])$. This result clarifies the scope and limits of Gaussian universality for ERMs. Additionally, we prove that any $\mathcal{C}^2$ regularizer is asymptotically equivalent to a quadratic form determined solely by its Hessian at zero and gradient at $μ_{\hatθ}$. Numerical simulations across diverse losses and models are provided to validate our theoretical predictions and qualitative insights.


Instance-SpecificAsymmetricSensitivityin DifferentialPrivacy

Neural Information Processing Systems

While the inverse sensitivity mechanism was shown to be instance optimal, it was only with respect to a class of unbiased mechanisms such that the most likely outcome matches the underlying data.


Russia-Ukraine war: List of key events, day 1,457

Al Jazeera

How the US left Ukraine exposed to Russia's winter war Will Europe use frozen Russian assets to fund war? How can Ukraine rebuild China ties? Russian forces launched 448 attacks on 34 settlements in Ukraine's front-line Zaporizhia region in a single day, injuring a six-year-old child and damaging homes, cars and other infrastructure, regional governor Ivan Fedorov wrote on the Telegram app. Russian drone, missile and artillery attacks on Ukraine's Kherson region injured five people and damaged homes, including seven high-rise buildings, the local military administration said on Telegram. Russian attacks also continued in Ukraine's Dnipropetrovsk and Sumy regions, but local officials there noted that "fortunately, no people were injured".




ANPL: Towards Natural Programming with Interactive Decomposition Di Huang

Neural Information Processing Systems

Though LLMs are capable of generating plausible programs, it's challenging to interact with the LLMs further to revise the program, especially if the user's specific requirements are different from the initial proposal.